Modeling Conventionalization and Predictability within MWEs at the Brain Level

While expressions have traditionally been binarized as compositional and noncompositional in linguistic theory, Multiword Expressions (MWEs) demonstrate finer-grained distinctions. Using Association Measures like Pointwise Mutual Information and Dice\u27s Coefficient, MWEs can be characterized as having different degrees of conventionalization and predictability. Our goal is to investigate how these gradiences could reflect cognitive processes. In this study, fMRI recordings of naturalistic narrative comprehension is used to probe to what extent these computational measures and the cognitive processes they could operationalize are observable during on-line sentence processing. Our results show that Dice\u27s Coefficent, representing lexical predictability, is a better predictor of neural activation for processing MWEs. Overall our experimental approach demonstrates how we can test the cognitive plausibility of computational metrics by comparing it against neuroimaging data

Differentiating Phrase Structure Parsing and Memory Retrieval in the Brain

On some level, human sentence comprehension must involve both memory retrieval and structural composition. This study differentiates these two processes using neuroimaging data collected during naturalistic listening. Retrieval is formalized in terms of multiword expressions while structure-building is formalized in terms of bottom-up parsing. The results most strongly implicate Anterior Temporal regions for structure-building and Precuneus Cortex for memory retrieval

A Neurolinguistic Approach to Noncompositionality and Argument Structure

Understanding the neural bases of language comprehension is to understand the implementation of language processing in the brain and how it affects language performance. Within a neurolinguistic study, we can examine the connection between linguistic competence and language performance at the cerebral level and whether the distinctions that we draw in linguistic theory map on to particular brain systems. Recently there has been an increase in psycholinguistic and neurolinguistic research using naturalistic stimuli following Willem’s (2015) encouragement to investigate the neural bases of language comprehension with greater ecological validity. Along with naturalistic stimuli, applying tools from computational linguistics to neuroimaging data can help us gain further insight into naturalistic, online language processing as computational modeling makes it easier to study the brain responses to contextually situated linguistic stimuli. (Brennan 2016). Utilizing this approach, in this dissertation I focus on two topics: noncompositional expressions (MWEs) and verbal argument structure. Across seven studies, I show how we can utilize various models and metrics from computational linguistics to operationalize cognitive hypotheses and help us better understand the neurocognitive bases of language processing. This dissertation is based on a large-scale fMRI dataset based on 51 participants listening to Saint-Exupéry's The Little Prince (1943), comprising 15,388 words and lasting over an hour and a half. While previous work has examined individual types of noncompositional expressions (such as idioms, compounds, binomials), this work combines this heterogeneous family of word clusters in a single analysis. Association measures are metrics from corpus and computational linguistics to identify collocations. This research contributes a gradient approach to these noncompositional expressions by repurposing association measures and demonstrates how they can be adapted as cognitively plausible metrics for language processing, among other findings. This dissertation also investigates the neural correlates of argument structure and corroborates previous controlled, task-based experimental work on the syntactic and semantic constraints between a verb and its argument. Another finding is that the Precuneus, not traditionally considered a core part of the perisylvian language network, is involved in both processing noncompositional expressions and diathesis alternations for a given verb. Overall, based on this interdisciplinary approach, this dissertation presents empirical evidence through neuroimaging data, linking linguistic theory with language processing

Data for: Eelbrain: A Python toolkit for time-continuous analysis with temporal response functions

This dataset accompanies “Eelbrain: A Python toolkit for time-continuous analysis with temporal response functions” (Brodbeck et al., 2021) and is a derivative of the Alice EEG datasets collected at the University of Michigan Computational Neurolinguistics Lab (Bhattasali et al., 2020), licensed under CC BY (https://creativecommons.org/licenses/by/4.0/) and the original work can be found at DOI: 10.7302/Z29C6VNH. The files were converted from the original matlab format to fif format in order to be compatible with Eelbrain. This dataset includes the EEG data for 33 participants, which were used in the example analyses for the paper. The original Alice dataset included data from all 49 participants and participants were excluded due to artifacts and incorrect behavioral responses (for more details see Bhattasali et al., 2020). You can use the Python script data_grab.py at https://github.com/christianbrodbeck/Alice-Eelbrain to download and unzip these files into a specified destination folder

Modeling Conventionalization and Predictability in Multi-Word Expressions at Brain-level

International audienceLinguistic expressions have been binarized as compositional and non-compositional given the lack of composionallinguistic analysis, Multi-word Expressions (MWEs) demonstrate finer-grained degrees of conventionalization and predictability in psycholinguisitcs, which canbe quantified through computational Association Measures, like Point-wise Mutual Information and Dice's Coefficient.In this study, fMRI recordings of naturalistic narrative comprehension is used to investigate to what extent these computational measures and the underlying cognitive processes they could reflect are observable during on-line naturalistic sentence processing

Neuro-computational models of language processing

International audienceEfforts to understand the brain bases of language face the mapping problem: at what level do linguistic computations and representations connect to human neurobiology? We review one approach to this problem that relies on rigorously defined computational models to specify the links between linguistic features and neural signals. Such tools can be used to estimate linguistic predictions, model linguistic features, or specify a sequence of processing steps that may be quantitatively fit to neural signals collected while participants use language. Progress has been helped by advances in machine learning, attention to linguistically interpretable models, and openly shared datasets that allow researchers to compare and contrast a variety of models. We describe one such dataset in detail in the supplementary materials

Neural Correlates of Object-Extracted Relative Clause Processing Across English and Chinese

Abstract Are the brain bases of language comprehension the same across all human languages, or do these bases vary in a way that corresponds to differences in linguistic typology? English and Mandarin Chinese attest such a typological difference in the domain of relative clauses. Using functional magnetic resonance imaging with English and Chinese participants, who listened to the same translation-equivalent story, we analyzed neuroimages time aligned to object-extracted relative clauses in both languages. In a general linear model analysis of these naturalistic data, comprehension was selectively associated with increased hemodynamic activity in left posterior temporal lobe, angular gyrus, inferior frontal gyrus, precuneus, and posterior cingulate cortex in both languages. This result suggests the processing of object-extracted relative clauses is subserved by a common collection of brain regions, regardless of typology. However, there were also regions that were activated uniquely in our Chinese participants albeit not to a significantly greater degree. These were in the temporal lobe. These Chinese-specific results could reflect structural ambiguity-resolution work that must be done in Chinese but not English object-extracted relative clauses

Neural correlates of semantic number: A cross-linguistic investigation

International audienceOne aspect of natural language comprehension is understanding how many of what or whom a speaker is referring to. While previous work has documented the neural correlates of number comprehension and quantity comparison, this study investigates semantic number from a cross-linguistic perspective with the goal of identifying cortical regions involved in distinguishing plural from singular nouns. Three fMRI datasets are used in which Chinese, French, and English native speakers listen to an audiobook of a children's story in their native language. These languages are selected because they differ in their number semantics. Across these languages, several well-known language regions manifest a contrast between plural and singular, including the pars orbitalis, pars triangularis, posterior temporal lobe, and dorsomedial prefrontal cortex. This is consistent with a common brain network supporting comprehension across languages with overt as well as covert number-marking

Localising Memory Retrieval and Syntactic Composition: An fMRI Study of Naturalistic Language Comprehension

International audienceThis study examines memory retrieval and syntactic composition using fMRI while participants listen to a book, The Little Prince. These two processes are quantified drawing on methods from computational linguistics. Memory retrieval is quantified via multi-word expressions that are likely to be stored as a unit, rather than built-up compositionally. Syntactic composition is quantified via bottom-up parsing that tracks tree-building work needed in composed syntactic phrases. Regression analyses localise these to spatially-distinct brain regions. Composition mainly correlates with bilateral activity in anterior temporal lobe and inferior frontal gyrus. Retrieval of stored expressions drives right-lateralised activation in the precuneus. Less cohesive expressions activate well-known nodes of the language network implicated in composition. These results help to detail the neuroanatomical bases of two widely-assumed cognitive operations in language processing